Papers with cross-modal representation learning
Multi-Scale Spectral Selection and Entropy-Guided Uncertainty Fusion for Multimodal Rumor Detection (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for multimodal content detection fail to capture cross-modal semantic inconsistencies and ignore inherent noise in multimodal features. |
| Approach: | They propose a multimodal rumor detection method based on a frequency domain spectral selection method and entropy-guided uncertainty fusion method to capture cross-modal semantic inconsistencies. |
| Outcome: | The proposed method outperforms state-of-the-art methods in multimodal rumor detection . it shows stronger detection capability and robustness on multiple datasets . |
Exploring Compositional Image Retrieval with Hybrid Compositional Learning and Heuristic Negative Mining (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing CIR models are pre-trained on uni-modal data, resulting in unimodal data. |
| Approach: | They propose a CIR model HyCoLe-HNM with CLIP as the backbone . they use a gated fusion mechanism from a question answering model to perform compositional learning . |
| Outcome: | The proposed model achieves state-of-the-art performance on three CIR datasets . it borrows a gated fusion mechanism from a question answering model to perform compositional fusion . |